Pagoda: A Model for Autonomous Learning in Probabilistic Domains

ley as a test bed for designing intelligent agents. The system consists of an overall agent architecture and five components within the architecture. The five components are (1) goaldirected learning (GDL), a decisiontheoretic method for selecting learning goals; (2) probabilistic bias evaluation (PBE), a technique for using probabilistic background knowledge to select learning biases for the learning goals; (3) uniquely predictive theories (UPTs) and probability computation using independence (PCI), a probabilistic representation and Bayesian inference method for the agent’s theories; (4) a probabilistic learning component, consisting of a heuristic search algorithm and a Bayesian method for evaluating proposed theories; and (5) a decision-theoretic probabilistic planner, which searches through the probability space defined by the agent’s current theory to select the best action. PAGODA is given as input an initial planning goal (its overall behavioral goal, for example, “maximize utility”) and probabilistic background knowledge about the domain. The agent selects learning goals (features in the world about which predictive theories will be formed) that will maximize the agent’s ability to achieve its planning goal. The learner uses the probabilistic background knowledge to select learning biases for each learning goal and performs a heuristic search through the space of UPTs, using Bayesian techniques to evaluate the theories. The planner uses the best current theory to choose actions that satisfy its planning goal and to generate new learning goals for the inductive learning subsystem to focus on. My Ph.D. dissertation (desJardins 1992a)1 describes PAGODA (probabilistic autonomous goal-directed agent), a model for an intelligent agent that learns autonomously in domains containing uncertainty. The ultimate goal of this line of research is to develop intelligent problem-solving and planning systems that operate in complex domains, largely function autonomously, use whatever knowledge is available to them, and learn from their experience. PAGODA was motivated by two specific requirements: The agent should be capable of operating with minimal intervention from humans, and it should be able to cope with uncertainty (which can be the result of inaccurate sensors, a nondeterministic environment, complexity, or sensory limitations). I argue that the principles of probability theory and decision theory can be used to build rational agents that satisfy these requirements. PAGODA incorporates innovative techniques for using the agent’s existing knowledge to guide and constrain the learning process as well as a powerful new mechanism for representing, reasoning with, and learning probabilistic knowledge. Additionally, PAGODA provides a conceptual framework for addressing important open problems such as incremental, resource-bounded learning and knowledge-based learning and planning. PAGODA was implemented in the RALPH (rational agent with limited processing hardware) world, a simulated two-dimensional world used at the University of California at BerkeGoal-Directed Learning